JBrowse 安装和使用

Posted on | Edited on

这件事情弄了我两天。查看处理错误信息占据了大部分时间。好在最后终于成功了。

感悟，对于技术文档不能完全照搬。中文的技术文档，主要的作用是了解，最后的实践一定要充分参考官方的英文文档。

基因浏览器介绍（入门）

https://www.plob.org/article/11742.html

在日常数据处理和分析工作中，根据分析项目的不同我们会面对各种各样的文件，比如mapping得到的SAM或者BAM文件，在此基础上转化而来的记录各种（定量）信息的bedgraph文件、Wig文件或者tdf文件，亦或记录variant信息的VCF文件。以至于有一种比较调侃的说法，所谓“生物信息”就是整天和各种格式的文件打交道，转换来转换去。

JBrowse 是今天要介绍的主角。它是GMOD开源项目的一部分，想了解这个开源项目可以查看它的官方网站http://gmod.org/wiki/Main_Page

完全基于HTML5和Javascript构建的JBrowse通过AJAX技术实现了数据的异步加载，所以响应速度非常快。由于Javascript将大量的计算工作在前端完成，服务器端只需要向浏览器客户端发送静态文件，因此也极大程度减轻了服务器端的负担。

完全基于HTML5和Javascript构建的JBrowse通过AJAX技术实现了数据的异步加载，所以响应速度非常快。由于Javascript将大量的计算工作在前端完成，服务器端只需要向浏览器客户端发送静态文件，因此也极大程度减轻了服务器端的负担。

准备参考序列

安装成功之后首先要做的是格式化参考序列，支持的格式有fasta， gff，或者是用samtools faidx处理过的indexed fasta 文件。参考序列生成的track 会为后续所有文件提供一个坐标，一直放大后参考序列的碱基也会显示出来。

通常我们使用某一个物种的genome fasta 文件，默认情况下，每一条染色体会独立为一条参考序列。如果想把RNA或者蛋白序列当成参考序列也没有问题。

需要注意的是所有数据默认都会输出在out目录中，如果你想用JBrowse展示不同物种的信息，最好使用--out参数指定单独的目录。

在这里提前说明，后面所有基于命令行设置的配置信息都会自动生成在tarckList.json文件中。

准备参考序列需要用到的是bin目录下的prepare-refseqs.pl脚本

对于VCF文件，需要提前做的准备就是使用bgzip压缩之后再利用tabix -p vcf 建立index。

展示方式和BigWig类似，其中type要定义为 “JBrowse/View/Track/HTMLVariants”；而storeClass定义为”JBrowse/Store/SeqFeature/VCFTabix” 即可。

尝试安装

主要参考这篇文献

https://yq.aliyun.com/articles/650480

没有成功，应该是perl模块的安装问题。不知道是否和conda有关。

! Installing GD failed. See /root/.cpanm/work/1551106022.8560/build.log for details. Retry with –force to force install it.
Searching XML::LibXML (0) on cpanmetadb …
Already tried XML-LibXML-2.0134. Skipping.
! Installing the dependencies failed: Module ‘XML::LibXML’ is not installed, Module ‘XML::LibXML::Reader’ is not installed, Module ‘GD’ is not install
ed, Module ‘DBI’ is not installed, Module ‘DB_File’ is not installed
! Bailing out the installation for BioPerl-1.7.5.
Searching JSON::XS (0) on cpanmetadb …
Unpacking JSON-XS-4.01.tar.gz
FAIL

! Installing JSON::XS failed. See /root/.cpanm/work/1551106022.8560/build.log for details. Retry with –force to force install it.
Searching Bio::Root::Version (1.006000) on cpanmetadb …

-> FAIL Installing the dependencies failed: Module ‘DBI’ is not installed, Module ‘Heap::Simple::XS’ is not installed, Module ‘Bio::Annotation::Simple
Value’ is not installed, Module ‘Bio::SeqFeature::Annotated’ is not installed, Module ‘DB_File’ is not installed, Module ‘Bio::Index::Fasta’ is not in
stalled, Module ‘Bio::SeqFeature::Lite’ is not installed, Module ‘DBD::SQLite’ is not installed, Module ‘Devel::Size’ is not installed, Module ‘Bio::F
eatureIO’ is not installed, Module ‘JSON::XS’ is not installed, Module ‘Bio::Root::Version’ is not installed, Module ‘PerlIO::gzip’ is not installed,
Module ‘Bio::OntologyIO’ is not installed
-> FAIL Bailing out the installation for JBrowse-.

1
2
3
4
5
6
7
8
9
perl -MCPAN -e shell
install DBI

conda install -c bioconda perl-module-list
conda install -c bioconda perl-dbi
conda install -c bioconda perl-bioperl 

instmodsh
locate XML/SAX.pm

1551113280461

/home/conda/feedstock_root/build_artifacts/perl_1548813468557/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc: No such file or directory
make: *** [Perl.o] Error 127

docker安装（没有成功）

http://www.runoob.com/docker/centos-docker-install.html

安装docker很顺畅

1
2
3
4
5
6
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sudo yum makecache fast
sudo yum -y install docker-ce
sudo systemctl start docker
docker run hello-world

由于本地没有hello-world这个镜像，所以会下载一个hello-world的镜像，并在容器内运行。

jbrowse/gmod-jbrowse 卡住了

1
2
docker run -p 8080:80 jbrowse/jbrowse-1.12.0
# 有问题

conda 安装（安装成功了，不知道如何启用）

1
2
# 需要超级用户权限，不然会失败
conda install -c bioconda jbrowse
1
2
3
# conda 的卸载命令
conda list | grep jbrowse
conda remove jbrowse

安装Nginx

yum install nginx

[root@localhost qi]# netstat -ntpl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:111 0.0.0.0: LISTEN 1/systemd
tcp 0 0 0.0.0.0:80 0.0.0.0: LISTEN 1823/nginx: master
tcp 0 0 0.0.0.0:6000 0.0.0.0: LISTEN 6706/X
tcp 0 0 192.168.122.1:53 0.0.0.0: LISTEN 6629/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0: LISTEN 6296/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0: LISTEN 6299/cupsd
tcp 0 0 127.0.0.1:6010 0.0.0.0: LISTEN 13051/sshd: qi@pts/
tcp 0 0 127.0.0.1:6011 0.0.0.0: LISTEN 23741/sshd: qi@pts/
tcp 0 0 127.0.0.1:6012 0.0.0.0: LISTEN 31798/sshd: qi@pts/
tcp6 0 0 :::3306 ::: LISTEN 7024/mysqld
tcp6 0 0 :::111 ::: LISTEN 1/systemd
tcp6 0 0 :::80 ::: LISTEN 1823/nginx: master
tcp6 0 0 :::6000 ::: LISTEN 6706/X
tcp6 0 0 :::22 ::: LISTEN 6296/sshd
tcp6 0 0 ::1:631 ::: LISTEN 6299/cupsd
tcp6 0 0 ::1:6010 ::: LISTEN 13051/sshd: qi@pts/
tcp6 0 0 ::1:6011 ::: LISTEN 23741/sshd: qi@pts/
tcp6 0 0 ::1:6012 ::: LISTEN 31798/sshd: qi@pts/ 

1
2
kill 1823
service nginx star

Centos7 yum配置Nginx

/home/qi/miniconda3/pkgs/jbrowse-1.16.2-pl526h6bb024c_6/opt/jbrowse

又卸载yum remove nginx

成功的操作

  1. 将web服务器换成了apache，放文件的目录就是 /var/www/html

How to Install Apache on CentOS 7

CentOS 7 安装 Apache, MySQL, PHP 指南

  1. Jbrowse下载安装

下面这句话很关键，提到了不用 root 或 sudo 安装。我也不知道为什么，这部分：

Installing Perl prerequisites 成功。这是昨天一直没成功的地方

Formatting data 失败。提取报错信息里的命令，运行，获取错误信息。Can’t locate local/lib.pm

3. Run the automated-setup script, ./setup.sh, which will attempt to install all of JBrowse’s (modest) prerequisites for you in the jbrowse/ directory itself. Note that setup.sh should not be run as root or with sudo.

(py3.6) [qi@localhost JBrowse-1.16.3]$ ./setup.sh
Gathering system information …done.
NOTE: Legacy scripts wig-to-json.pl and bam-to-json.pl have been removed from setup. Their functionality has been superseded by add-bam-track.pl and add-bw-track.pl. If you require the old versions, please use JBrowse 1.12.3 or earlier.
Minimal release, skipping node and Webpack build
Installing Perl prerequisites …done.

Formatting Volvox example data … failed. See setup.log file for error messages.

Formatting Yeast example data … failed. See setup.log file for error messages.

Formatting Volvox example data …

  • rm -rf sample_data/json/volvox
  • bin/prepare-refseqs.pl –fasta docs/tutorial/data_files/volvox.fa –out sample_data/json/volvox
    Attempting to create directory /var/www/html/JBrowse-1.16.3/bin/../src/perl5/../../extlib
    mkdir sample_data/json/volvox: Permission denied at /var/www/html/JBrowse-1.16.3/bin/../src/perl5/JsonFileStorage.pm line 64.

Formatting Yeast example data …

  • rm -rf sample_data/json/yeast/
  • bin/prepare-refseqs.pl –fasta sample_data/raw/yeast_scaffolds/chr1.fa.gz –fasta sample_data/raw/yeast_scaffolds/chr2.fa.gzip –out sample_data/json/yeast/
    mkdir sample_data/json/yeast/: Permission denied at /var/www/html/JBrowse-1.16.3/bin/../src/perl5/JsonFileStorage.pm line 64.

Can’t locate local/lib.pm in @INC (@INC contains: /var/www/html/JBrowse-1.16.3/bin/../src/perl5/../../extlib/lib/perl5/5.16.3/x86_64-linux-thread-multi /var/www/html/JBrowse-1.16.3/bin/../src/perl5/../../extlib/lib/perl5/5.16.3 /var/www/html/JBrowse-1.16.3/bin/../src/perl5/../../extlib/lib/perl5/x86_64-linux-thread-multi /var/www/html/JBrowse-1.16.3/bin/../src/perl5/../../extlib/lib/perl5 /var/www/html/JBrowse-1.16.3/bin/../src/perl5 /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /var/www/html/JBrowse-1.16.3/bin/../src/perl5/JBlibs.pm line 25.
Compilation failed in require at bin/prepare-refseqs.pl line 5.
BEGIN failed–compilation aborted at bin/prepare-refseqs.pl line 5

1
2
3
4
5
6
7
locate local/lib.pm
cp -r /home/qi/perl5/lib/perl5/local /var/www/html/JBrowse-1.16.3/bin/../src/perl5/../../extlib/lib/perl5/5.16.3


sudo bin/prepare-refseqs.pl --fasta docs/tutorial/data_files/volvox.fa --out sample_data/json/volvox

sudo bin/prepare-refseqs.pl --fasta sample_data/raw/yeast_scaffolds/chr1.fa.gz --fasta sample_data/raw/yeast_scaffolds/chr2.fa.gzip --out sample_data/json/yeast/

地址：

http://10.164.6.154/JBrowse-1.16.3/index.html?data=sample_data/json/volvox

2019-2-28 19:53:01

今天登入ip的时候，没有显示。

Enable Apache to start at boot, and restart the service for the above changes to take effect:

1
2
sudo systemctl enable httpd.service
sudo systemctl restart httpd.service
1
2
3
4
# 确认http服务是否开启，端口 80
sudo systemctl is-enabled httpd.service
# 查看历史记录，awk去掉第一列，sed去掉行首空格
history | tail -n 40 | awk '{$1="";print $0}' | sed 's/^ *//'

准备参考序列和特征数据

JBrowser 用的是 perl 5.16.3

所有track都生成在默认的data目录中

1
2
3
4
5
6
# 指定perl为/usr/bin/perl 

# seq  trackList.json  tracks.conf
/usr/bin/perl bin/prepare-refseqs.pl --fasta ~/data2/tan_rna/genome/HWB.chromosome.fa
# tracks
/usr/bin/perl bin/flatfile-to-json.pl --gff HWB.gene.models.gff3 --trackType CanvasFeatures --trackLabel HWBgff

不指定会出现如下错误：[^1]

(base) [qi@localhost JBrowse-1.16.3]$ bin/flatfile-to-json.pl –help
perl: symbol lookup error: /var/www/html/JBrowse-1.16.3/bin/../src/perl5/../../extlib/lib/perl5/x86_64-linux-thread-multi/auto/JSON/XS/XS.so: undefined symbol: Perl_xs_apiversion_bootcheck

查资料

This type of error is almost always indicates you are loading a module that was installed using a different build of Perl.

https://stackoverflow.com/questions/51375821/perl-xs-apiversion-bootcheck

[^1]: 不指定，如果用sudo 也不会出现错误，如：sudo bin/prepare-refseqs.pl --fasta ~/data2/tan_rna/genome/HWB.chromosome.fa --out sample_data/json/HWB 不清楚原因

建立索引方便搜索

缺这一步，搜索功能不能用

1
2
# 生成 ./data/names
/usr/bin/perl bin/generate-names.pl

http://10.164.6.154/JBrowse-1.16.3/index.html?data=data

VCF tracks

http://jbrowse.org/docs/variants.html

1
2
3
for i in *.vcf; do bgzip $i; done

for i in *.vcf.gz; do tabix -p vcf $i; done